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Dear Readers, 


Today we are proud to deliver a special 
BSD Mag issue with compilation of all 
the best David Carliers articles. David 
has been cooperating with us for a cou- 
ple of years. During this time he has de- 
livered a great number of very technical 
articles, with thousands lines of code. 
His articles have always been in line with 
issue themes and we have a feeling that 
whatever topic we would ask for, he is 
able to write about it. He possesses 
great knowledge and passion to open 
source systems, mostly BSD - of course. 


We have chosen his best articles about 
FreeBSD, NetBSD, NodeJS, Hard- 
nedBSD, FreeBSD Security and Cloud. 


If you have been wondering who David 
actually is, at the end of this issue you 
will find his bio. 


We hope you will enjoy the reading. 
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Among all the numerous specific features of 
FreeBSD, there is a famous command line to 
dump the statistics of the various current proc- 
esses, procstat. Its internal API is fortunately 
exposed via the well named libprocstat library. 
Let's imagine we want to display it via a web 
page so for this article, we re going to use 
CppCms, one of the good quality C++ web de- 
velopment frameworks with a current 
FreeBSD 10.2 release version. 
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If you usually program in Linux and you are 
considering a potential switch to FreeBSD, 
this article will give you an overview of the pos- 
sibilities. 


The Journey of a C Developer in 
FreeBSD’s World 23 


Moving from Linux to FreeBSD involves quite 
a number of changes; some gains and some 
losses. As a developer, for most of the pro- 
gramming languages, especially the high level 
ones, there are no meaningful disturbing 
changes. But for languages like C (and its sib- 
ling C++), if you want to port your software, li- 
braries, etc., some points might need to be 
considered. 
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In this article, we will have an overview of 
what is called a syscall (system call short- 
ened), from the kernel side to the userland 
then in the end how to create a new one, in 
FreeBSD. It is assumed you know how to 
build FreeBSDcurrent and have some knowl- 
edge about C language. 
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In this article, we will give an overview of the 
nature of the FreeBSD’s kernel. The important 
configuration files will be explained in addition 
to learning how to compile the whole system 
with more options, with more debugging infor- 
mation. Very useful for kernel development. 
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For this mid-summer, we will approach a 
lighter subject, NetBSD and its ports system. 
Pkgsrc is the framework to build third party 
packages for this system. We will see how to 
create a package and hopefully submit it. 
Hence, the pkgsrc is supposedly already in 
your system. Otherwise, a full guide is avail- 
able here. 
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62 


The Java Debugger (JDB) is a simple 
command-line debugger for Java classes. The 
jdb command and its options call the JDB. 
The jdb command demonstrates the Java Plat- 
form Debugger Architecture (JDBA) and pro- 
vides inspection and debugging of a local, or 
remote, Java Virtual Machine (JVM). 
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A secure webserver on FreeBSD with Hia- 
watha 68 


In most cases, when it comes to choosing a 
web server, Nginx comes quickly to mind (lI 
personally appreciate this one a lot, no doubt 
about this). However, an interesting alterna- 
tive exists that embeds some very nice fea- 
tures, an alternative called Hiawatha. 
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Nodejs is well known to allow building server 
applications in full JavaScript. In this article, 
we'll see how to build nodejs from source 
code on FreeBSD. You will need autoconf 
tools, GNU make, Python, linprocfs enabled 
and libexecinfo installed. GCC/G++ compiler 
suite (C++11 compliant, ideally 4.8 series or 
above) or possibly clang can be used to com- 
pile the whole source. 
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Previously, we've seen how to build NodeJS 
from the sources in FreeBSD with minor 
source code changes. This time, we'll have an 
overview of the application’s build process. Nu- 
merous excellent tutorials exist to build a 
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ever, the possibility also exists to build an ap- 
plication natively in C/C++. It is exactly what 
we're going to see. 
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In this article, we will have an overview of writ- 
ing a cloud service. Various ways exist to 
achieve your goals but we will focus on one 
that is memory efficient, multiplatform (POSIX 

systems), multi language (from C++ to Er- 
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| recently fully wrote a cloud service and it 
worked reliably. 


To illustrate this, we will make a basic remote 
file handler, the server is written in C++ and 
the client written in Python as an example. 
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Using the FreeBSD's procstat API in a 
web context 


Among all the numerous specific features of FreeBSD, 
there is a famous command line to dump the statistics of 
the various current processes, procstat. Its internal API is 
fortunately exposed via the well named libprocstat library. 
Let's imagine we want to display it via a web page so for 
this article, we re going to use CppCms, one of the good 
quality C++ web development frameworks with a current 
FreeBSD 10.2 release version. 


1. Procstat API 
The list of the available functions can be viewed in this page 


https:/www.freebsd.org/cgi/man.cgi?query=libprocstat&sektion=3&apropos=0&manpath=FreeB 
SD%2010.0-RELEASE 


We just need to include the necessary headers and to link our application to the shared library 
libprocstat, simply. For our basic procstat service, we will expose the pids, the paths of the proc- 
esses and the owners of those. 


2. CppCms 


We could have used an usual full PHP solution, calling procstat utily via a system call, possibly 
parsing the output and displaying it. However, doing web development via low level langages is 
also possible especially in the embedded environments where the resources usage count. 


CppCms has a package, so pkg install copcms (or via the ports) is sufficient. This framework 
has a lot of useful features, session handling, caching, native encoding handling. For our basic 
usage, we'll use their advanced template system with the addition of jQuery to make it more ap- 
pealing. 
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Let's start with the template's content. For this purpose we need a C++ prototype and a CopCms 
template file. 
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ProcinfoContentSkin.tmpl: 


// For who has experienced various templates solution for Java, PHP and so on, some parts 
seem pretty familiar 


c++ #include "proclist.h" %> => We include simply our C++ prototype here 


<% skin ProcinfoContentSkin %> => Useful when the template are shared libraries 
<% view ProcinfoContent uses content::ProcinfoContent %> 
<o template render() %> 
Sialic lee. 
<head> 


<link rel="stylesheet" 
href="//jqueryui.com/jquery-wp-content/themes/jquery/css/base.css?v=l 
i 


<link rel="stylesheet" 


href="//jqueryui.com/jquery-wp-content/themes/jqueryui.com/style.css" 
S 


SSo(eicuhone Boise) 7 (ererol=)| Mae iNisiaen @1eiiN// gpehitatan a lm NOI Aan yrs ee Goreng iene. 


fee 1a alene 
Bie COUe. | quem yCoOm, Un Meh A ole ome ow) Ser oe 


<script type="text/javascript"> 
Sig aL Guaieucbrenal (leant 
Sa GMclclere nysik)) sacvena wclollkam() 
("tbody"), dasableselection (7; 
a 
GiGi ener, 


</head> 
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<body class="jquery-ul page-template-default"> 
<hl>Processes statistics</hl1> 
Colby, Jenico Meleneueslatialsia >. 
<div id="content-wrapper"> 
<div id="content"> 
<table class="ui-sortable"> 
Bea 
ilove 2 IE De< ciel: 
<th PATH= (tii 
<th>ARGUMENTS</th> 
<th>OWNER</th> 
<<) bie 


——EDody— 


<% foreach info in pinfos %> => Iterate through the pinfos member of 
the content's class ... 


<tr> => ... then ‘echoing’ each field of a Procinfo struct 


<td class="ui-state-default ui-sortable-handle"><%= 
epics 7 bel 


<td class="ui-state-default ui-sortable-handle"><%= 
.pathName %></td> 


<td class="ui-state-default ui-sortable-handle"><%= 
penicieem ee Aero) 


<td class="ui-state-default ui-sortable-handle"><%= 
UserName. (<—.—siIntoO,userEuliNamoe. ja—.—siInto,useriome:.>—/7 car 
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PU MehiCde 10s ar eam 
Lee dubiel= <<a simine ci 


camer At t@les Gis evel Uakiokala es 


Sleeve i(inaighe: 
<sys/param.h> 
<sys/queue.h> 
<sys/socket.h> 
Asn iey/ eaircreie la laes 


<sys/types.h> 





<sys/user.h> 


#include <pwd.h> 


#include <libprocstat.h> 


#include "proclist.h" 


ell cision JeNgele(sieche G2 oblollaiise [sjojoleniis i arel 0) ollimberheukounl. 
private: 

JQnmelersrerche = ers 

@lojahe =i oh ass snao ella te Crolh eave moe, 
jejotedialen 


Nao GTSMersn a (Glee) e) (ans us (aan lai @l anna ss anye)) wk gO) O18) 6) (is: 


apple arLon ves i) 
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// We're opening the processes info via the internal sysctl system 
// There are other ways, via a kernel's core dump file or via kvm ... 
[Ske a easiereNee ne Ree \ergl ies fener ed LG) ee 

} 
EOC sae aay 
SIG Olen ersyr, elo sen eies)p 


i 


aliceibloWk ey omkel auitcbiedsueels qisveredigles abucll) ia 


Se OLN Sem Keleiancie: Aveo acl, fee isrcre ave Cl “eh a 


cloigsie Jcimghie)enaerer )\<2) ISM MMUEL Ua Oca NE ey sie <Ielo@ve ie, @.ciMguble sjenaere 


eel ee 


remgisie. Laeuae jcuaere! a ).do) ge MUIMUBIeNa Ig oN Verse (clogs: Cligiblon jonaeve 
raed i 


alee WAI ole eal ei CnHel RG ola s.ck uenkol) 


iat =e] a gle 


igiedzabneigie ike 
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void 
Ia OL Ove Neve Cian (rohiGuigcrU Ohm ms nenaabaahe® Le | 


UNS Toned ants eu; 


ali@iiG ake 


// we just get the processes information w/o their thread IDS though ... 


/! We could get also only a specific group of processes per TTY or user etc ... 


GI@UIBC) JONaelen Sl <joy — oldolersyercne, (ef= Bjoraelers (ersi 
SHOE) ee 


KERN PROES PROG, on 
1f (kp == NULL) 
Igoueiehel ge: 


De.piInlos. — 


rot BIG eae 1G (© 1a Son aS @ lM e Pas Oe 


qsort (kp, ct, sizeof(*kp), kp compare); //As the processes list is not 
ordered, we do per PID 


Bip le. Gee UW ne 2) ea 
char path [PATH MAX]; 


Oi eolereiee Lefer eyeneimgker is (eisiy Vel Gelli || jerehwlaly, “siya eum (e)cueigy) 9) 7 


is elem (oa) eel) 
Za @roubinwErer 4Oaky 
ISB S CCE Se deer ab || 3 rk ouie le 
(Suk sfereneioul elite “—suec Mass ene mace (erenclally, 
SEds "SErIngstream ss; 


// Here we get the possible arguments the process were called with ... 
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// args NULL terminated list pointer will be freed by procstat_close later 


Siaahig, “Sawsnecish Sh) Sigeleis bene WCpeubrenciejw \jekstya. les Wala 


Glgsiies wono~ialeps, —— eliaersy 


// pargs[0] == path here, so it is bypassed (hence we could have just used 
procstat_getargv ...) 


while (*++pargs) 


SUS Ge OT OG a Pig ciate ost 


1G AlroWGl6 Fofe laa US IBIC My aisibia Inahe A reyonstouece 4())e))a 
passwd pw, *res; 
memset (&pw, 0, sizeof (pw)); 
Gigehe deyrhe MOVs 
// Just to get more “human readable” process' user info 


ahde VAVeloiw one mboh jad C qexiiala| Alene ve biiyel sa asjonee oie 
sizeof (buf), &res) == 0) { 


pi.userName = 
SUB eln rouse gies (Geni Sie), savelitles by 


pi.userFullName = 
SIB(Ol 4 Brswe iealgereMmreie WSieleKors:)) F 


jcubaiblels velo (O)iilen =" seiwiely sesvenesl (er \eniejonty elaine) 


(Glee Onl gi Shere ele Meiaiuelsler <i erg) 


} 





CppCms uses the popular JSON format for the configuration's file as follow for our example. 
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The possibilities of configuration are pretty rich, here we're using the internal web server but in 
production, it might be preferable to configure in FastCGI mode and allowing a guenine web 
server, like Nginx, handling the client's connections. 
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If we planned to compile the template as a shared library, we would need also to declare it in our 
config. For more precise information, please read this page. 


First, we need to “compile” the template file into a C++ code via a CppCms utilily. 


Then compiling altogether our CppCms' application with this template. Indeed, for the sake of the 
simplicity and as we have only one template, we compile it statically. 


| would advise to use at least a Makefile ... The booster's library is necessary for the template's 
system otherwise it is also possible to render a HTML content directly in the application's level via 
an usual C++ stream like here. 


Once compiled, we can finally launch our CppCms's application. 
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./cppems procstat -c config.json 


Illustration SEQ "Illustration" \*Arabic 1: Here our sortable list of processes 


This is it, we can now read the processes list and rearrange the order in a fancy manner. There is 
a lot of room for improvements, hopefully, that might give some ideas to you, readers. | hope at 
least, that will give you also the curiosity to dig in more in the procstat's API. 
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If you usually program in Linux and you are considering a 
potential switch to FreeBSD, this article will give you an 
overview of the possibilities. 


FreeBSD comes with either applications from binary packages or compiled from sources (ports). 
They are arranged by software types (programming languages mainly in lang (or java specifically 
for Java), libraries in devel, web servers in www...) and the main tool for modern FreeBSD ver- 
sions is pkg, similar to Debian apt tools suite. Hence, most of the time, if you are looking for a spe- 
cific application/library, simply: 


without necessarily knowing the fully qualified name of the package, it is somehow sufficient. 


For example: 


pkg search php5 will display phpd5 itself and the modules, furthermore php56 specific version 
and so on. 


The main difference is, you are not forced to either choose the binary or the port but can have 
both if it suits your need, but keep in mind that compiling from source can take a certain amount 
of time to achieve, if that is an important point for you. If the ports tree is not already present on 
your server, portsnap fetch extract will fetch the ports tree for you by default in /usr/ports. Then, 
related to the software type described above, you just need to go to the related folder. For exam- 
ple, for installing php5 : 
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The second command, depending on which options you are going to choose, will display all the 
options available for each dependency (for example, if gd support is enabled, furthermore the op- 
tions for graphics/gd library will appear). 


However, most of the time, the binary packages are sufficient to cover most of the needs. 
2. Web development 


This is basically the easiest area to migrate to. Most Web languages do not use specific platform 
features, so most of the time, your existing projects might just be “drop-in” use cases. 


If your language of choice is PHP, luckily, this scripting language is workable in various operating 
systems, most of the Unixes and Windows. In the case of FreeBSD, you have even more different 
ports or binary package versions (5.4 to 5.6). In this particular case, you might need some spe- 
cific PHP modules enabled, luckily, they are available uatomatically or if the port is the way you 
chose, it is via the www/php5-extensions one. 





Terminal 
File Edit View Search Terminal Help 


php5-extensions-1.7 


bc style precision math functions f 


bzip2 library support 


CALENDAR 
CTYPE 
CURL 

DBA 

DOM 

EXIF 
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FTP 

GD 
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Figure 1. PHP port and modules. 


calendar conversion support 
ctype functions 

CURL support 

dba support 

DOM support 

EXIF support 

fileinfo support 

input filter support 
FTP support 

GD library support 
gettext Library support 
GNU MP support 


< (K > <Cancel> 





MAGAZINE 





BSD 


The FreeBSD Corner 





Of course, developing with Apache (both 2.2 and 2.4 series are available, respectively www/ 
apache22 and www/apache24 packages) or even better with Nginx (the last stable or the last de- 
velopment versions could be used, respectively www/nginx and www/nginx-devel packages) via 
php-fpm is possible. 


Outside of PHP, the same applies to Python / Django (www/py-django) and Ruby on Rails (www/ 
rubygen-rails), Python 2.7 and 3.5 (lang/python<version>) are available as Ruby until 2.2 (lang/ 
ruby<version>). 


In terms of databases, we have the regular RDBMS like MySQL and PostgreSQL (client and 
server are distinct packages: databases/ (mysql/portgresql)<version>-client and 
databases/ (mysql/postgresgql)<version>-server) and the more modern concept of 
NoSQL with CouchDB for example (databases/couchdb), MongoDB (databases/ 
mogodb), Cassandra (databases/cassandra) to name a few. 


Also, if you need to perform efficient Map / Reduce for Big Data work, you have either the well 
known Apache Hadoop or Apache Spark (respectively devel/hadoop and devel/spark). Last, if you 
ever need a search engine, Apache Solr/Lucene (textproc/apache-(solr/lucene)), Xapian 
(databases/xapian) and their various language bindings are available. 





















PhpProject1 - NetBeans IDE 8.0.2 


File Edit View Navigate Source Refactor Run Debug Profile Team Tools Window Help Q 


sal Fl eh & e <default> x) S- i > > ER: 

































































X|Files _| Ser. El (Start Page |” [aj index.php x 4) >\[~)[o) 
: ae Sane) History| BE B- QBGHBG PLR AG eo ZuBs H 
fad index.php g <?php ss 
© G@® Include Path 2 a 
a class BlogPost extends BaseBlogPost 
40{ 
5 public function __toString() 
6) F php PHP: Something Use’ x \ Y 
u return $this->getTitle(); Ei 9 
3 Ly € C65 php.net/manual/en/tutorial.useful.php 
10 
QB ?> 





Downloads Documentation | GetInvolved Help 


$HTTP * VARS arrays Instead, suchas $HTTP SERVER VARS . Although deprecated, these 
older variables still exist. (See also the note on old code.) 


To display this variable, you can simply do: 


Example #1 Printing a variable (Array element) 


<?php 
echo $_SERVER[ 'HTTP_USER_AGENT']; 
?> 


























Figure 2. PHP development under Netbeans. 
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Is it rather Java Web or any language based on the Java VM platform? In FreeBSD, you even 
have Java 8 (either java/openjdk8 and java/linux-oracle-jdk18), various popular frameworks and 
J2EE servers or servlet engines like Spring (java/springframework), Jboss (java/ 
jboss<version>), Tomcat (www/tomcat<version>), Jetty (www/jetty)... Even the more 
modern languages like Scala (lang/scala), Groovy (lang/groovy) can be found. 


Two languages described above, Python and Ruby, have their Java VM counterparts, Jython 
(lang/jython) and Jruby (lang/jruby), available as well, 


In term of Integrated Development Environment, there are still several choices. The venerable 
Netbeans (java/netbeans or java/netbeans-devel), Eclipse (java/eclipse ... side 
note, FreeBSD needs to have Kerberos support enabled, NO KERBEROS is /etc/make.conf 
or /etc/src.conf presence needs to be checked) with their numerous popular plugins. 


3. Low level development 


The BSD is shipped with a C and C++ compilers in base. In the case of FreeBSD 10.2, it is clang 
3.4.1 (in x86 architectures), otherwise modern versions of gcc, for developing with C++11, for ex- 
ample, are of course available too (lang/gcc<version> ... until gcc 5.2). 


Numerous libraries for various topics are also present, web services SOAP with gsoap through 
User Interfaces with GTK (x11-toolkits/gtk<version>), QT4 or QT 5 (devel/ 
qt<version>), malware libraries with Yara (security/yara) ... 


In term of IDEs, Eclipse and Netbeans described above allow both C/C++ development, Anjuta 
and Qtcreator are also available for important projects. If you prefer, FreeBSD has in base vi and 
Vi Improved can be found in ports / packages (editors/vim or editors/vim-lite without X11 support). 
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Java - Eclipse SDK 
File Edit Navigate Search Project Run Window Help 
s @Welcome ® * ce Be 


rar 
C) REECE Ce rm cd 
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© 


Next> | Cancel Finish 














Figure 3. Java Eclipse SDK. 


FreeBSD is a POSIX system, hence porting C/C++ code to this platform depends on the degree 
of portability of your projects, so the usage of specific “linuxisms” and such. 


In case more information is needed about porting software in FreeBSD and its specific tools, | 
would recommend the reading of BSDMag issues no 66 and 68. 


4. Android / Mobile development. 


In order to be able to do Android development, to a certain degree, the Linux compatibility layer 
(aka linuxulator) needs to be enabled. Also x11-toolkits/swt and linux-f10-gtk2 
port/package need to be installed (note that libswt-gtk-3550.so and 
libswt-pi-gtk-3550.so are needed, the current package is versioned as 3557, can be solved 
with symlinks). In the worst case, remember that bhyve (or Virtualbox) are available and can run 
any Linux distribution smoothly. 
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Packages Tools 


SDK Path: 

Packages 

w Name API Rev Status 

y¥i Tools a 

w¥# Android SDK Tools | /4.4.@ Installed 

¢# Android SDK Platform-toa '3.0..>5 Not installed 
+ Android SDK Build-tools | 3.0.) Not installed 
¢ Android SDK Build-tools ___'3.0.. Not installed 





Android SDK Manager Log 









Trees roo 
Installing ‘Samples. for SDK. API 23, revision 2 

Installed Samples for SDK API 23, revision 2 

Downloading Sources for Android SDK, API 23, revision 1 

Installing Sources for Android SDK, API 23, revision 1 

Installed Sources for Android SDK, API 23, revision 1 

Downloading Android TV ARM EABI v7a System Image, Android API 23, revision 2 
eae Android TV ARM EABI v7a System Image, Android API 23, revision 2 


nea Android TV ARM EABI v7a System Image, Android API 23, revision 2 Close 
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» 1G Android 4.1.2 (API 16 
> of Android 4.0.3 (API 15 
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awa ee = wr rawr 


Installing Android TV ARM EABI v7a System Image, Android API 23, revision 2 





Figure 4. SDK Manager under FreeBSD. 
5. Source Control Management. 


FreeBSD comes in base with a version of subversion, as FreeBSD source is in a subversion re- 
pository, prefixed svnlite, though, to avoid conflicts with the package/port. 


In addition, Git is present but via the package/port system with various options (with or without a 
user interface, subversion support). 


6. Conclusion 


FreeBSD has made tremendous improvements over the years to fill the gap with Linux whereas it 
still Keeps its own interesting specificities, hence there would not be too many blockers if your pro- 
jects are reasonably sized to consider a migration to FreeBSD. 
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Moving from Linux to FreeBSD involves quite a number of 
changes; some gains and some losses. As a developer, for 
most of the programming languages, especially the high 
level ones, there are no meaningful disturbing changes. But 
for languages like C (and its sibling C++), if you want to 
port your software, libraries, etc., some points might need 
to be considered. 


- How to move from Linux to FreeBSD - Basic knowledge of C programming 


¢ How to develop under FreeBSD 


As is often the case with C, it is not especially straightforward; the code itself might need some 
changes, minus the pure POSIX part. Let’s say your program needs to use some known network 
functions. 
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deaehe alia 
#include 
e aalave@nllblel=s 


ve tale euvele 


int 
(eign ab@be 


{ 


<sys/types.h> 
<sys/socket.h> 


<arpa/inet.h> 


eligejor delve “xsiatenye|| ||) 


iS Ga Ge eih@eis (eka pinale 


Glolghs wen telah a ore eh sepia dalle 


Nese ONG Siw ouuncne! (VLU MENTE Siljey asl gh) ei ait) 


fica Cee 


{ 


Bugler seldicns acwecwel | y) 


Suen ei shOge nabseia tet aie 


char hwaddr [6] 





In addition, FreeBSD provides a bunch of specific functions, like stricpy/stricat (safer versions of 
strcpy/strcat) and strtonum family functions, all of which are available in the base, whereas Linux 
must install the separate BSD library to have them. If you have any doubts about any functions, 
all manpages are available and very well written. 


FreeBSD is shipped by default with clang, whereas Linux relies on GCC suite. If you heavily use 
OpenMP, clang does not provide it yet so you might need to install GCC from ports. Somehow, 
clang mostly compiles faster and provides more informative warning and error messages. Fortu- 
nately, they share a significant amount of common flags. 


On Linux, you may use a custom memory allocator during your development, like jemalloc. It’s a 
very handy and useful library that allows you to generate statistics, to fill freed memory with spe- 
cific values, and to spot corrupted memory usage. 


Good news! You do not need to install it—FreeBSD libc’s malloc (aka phkmalloc) uses jemalloc 
internally. To print statistics from your application, for example, you need to include 
malloc np.h instead of jemalloc/jemalloc.h 
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As for the makefiles, this is the BSD format which differs from GNU style: 


A basic makefile for a library: 


A basic makefile for an application: 
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FreeBSD can handle GNU via (gnu)make, libtool, etc. via the ports. 


Or to save the effort of porting this part, it might be more handy to use cmake or scons. 


You might want to publish your library / application in pure FreeBSD’s path. You can make a port 
that can provide some options for the user. It can download the source and compile it with its de- 
pendencies in a natural manner. In addition, you can build a binary package to facilitate the distri- 
bution. 


john.doe@email.com 
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For instance, you can put the archive .tar.gz of the library in /usr/ports/distfiles, then 
type make checksum. Then, make install will compile and install it in /usr/local. The handbook 


of making ports is very useful to read. 


Furthermore, you can build a binary version of this port to facilitate its distribution. As simple as it 
is, pkg create myli. It will create a txz archive in the current folder. In the end, pkg _ install mylib 


will install it. 


Developing under FreeBSD is not the extreme challenge you might think it is. Even better, from 
coding to publishing, everything is thought out and made in a constant way without any external 
dependencies. If you want to go even further, like kernel development, again it is easy and in 
base. So there is no real reason to stay away from FreeBSD anymore, you are more than wel- 


come. 


The HardenedBSD project was created in 2014 by Oliver Pinter and Shawn Webb. The project 
aims to provide security enhancements to the FreeBSD project. We plan to upstream most, if not 


all, of our projects. 

The core HardenedBSD team consists of: 
¢ Oliver Pinter 

¢ Shawn Webb 

The developer team consists of: 

¢ David Carlier 
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e Nathan Dautenhahn 
¢ Danilo Egea Gondolfo 
¢ Oliver Pinter 


“Shawn Webb 


The following people and organizations have contributed to the HardenedBSD project: 
¢ Ilya Bakulin 

¢ Bryan Drewery 

¢ Danilo Egea Gondolfo 

¢ Dag-Erling Sm@rgrav 

¢ Robert Watson 

¢ Hunger 

¢ SoldierX - Donated a sparc64 and a BeagleBone Black 

¢ Hyper6 - Designed logo 


¢ Automated Tendencies - Substantial monetary donation. 
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In this article, we will have an overview of what is called a 
syscall (system call shortened), from the kernel side to the 
userland then in the end how to create a new one, in 
FreeBSD. 


It is assumed you know how to build FreeBSD-current and 
have some knowledge about C language. 


A syscall is a code implemented in the kernel side that can be potentially called from the userland 
for various purposes, network, generating some random data or controlling the system processes 
as well. Whatever the type of syscall, the userland never interacts directly with the kernel but via a 
defined interface. Let's start with a simple example, the getpagesize call, which is the number 
of bytes per page. 


Let's see how, in the kernel, it is implemented. 


Which simply returns the constant via the sysctl system. Let's see now how it is implemented in 
the userland side. 
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lib/libc/gen/getpagesize.c 


nah 

getpagesize (void) 

{ 
sige. 2iah only |p 
static int value; 
SUAS. We Gsbilwaeyr 


ghee snen mena 


if (value != QO) 


return (value); 


/* Here we try to get as much of the cached result as possible */ 


sieiiene 2 Mellie ehobe aligned le Cie ke recuible ssalvasiess (vecull las). 
if (error == 0 && value != 0) 
return (value); 
/* Otherwise we get the result via the sysctl call */ 
mib[0] CTL HW; 
iiaklon [kl HW PAGESIZE; 
Size = sizeof value; 
sib si Vesneub len Gite mrearsmr ale bl soa eaesuluya orem INO) Vis 


igs Wega eae albn We 





As you can see in this case, mainly the work is done in the userland side. Another type of syscall 
exists where all the implementation is done in the kernel side only but its symbol is transmitted via 
a symbol map. Next, let's have an eye on the getpid syscall. The usual C function has this 
signature. 


Those syscalls are organized per type in the kernel, implemented in the files 
src/sys/kern/(kern/sys)_ *.c. Basically, the sys_* files hold the function implementations 
but if the complexity requires it, some codes are split in the kern_* ones. 


First of all, the struct getpid_args reflects the userland parameters, every syscall needs a cor- 
responding *args struct to pass the userland parameters, hence as getpid has none. Here 
we have a dummy's. 


This type of syscall has this signature: 
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The thread parameter is the current thread from which we can get the current process, locking it 
in order to modify or just read a specific state. The integer returns here to, potentially, set errno in 
errors code paths. 


Here’s the corresponding getpid implementation. 


In the case of getpid, it is always successful, errno is never set so we return 0 directly. 
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PROG TEOCK (a); 
bab (Gee ech elcome o2 de bay NG la IDA I) 7 

(SHOE CL 2 9 Cle aCe Sate ee J SHG: 
PROC UNEOCK (po): 

} else { 
PROG SUNLOCK a); 
Spe esl ee Ga eraeleieicia amore <)) 7, 
[S/O ae | Otelel eat crerl 4 oleae le) I, 
ppid = pp->p_pid; 


ob.g Peibhe oie ets endereienscice Ak@rel.<)) 4: 


isc Hela | Gejomiel) 2 


/* Returns the real parent process whether it had exited or 


traverses the orphaned linked list */ 


SE EVag LCE mena en 


oreo! esculenta (sie reibisie Aoieere! aelaal kel) 


{ 


SOU GINICID ONASIO) UNC ey Nereus sie Hoy: 
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SPs se he (Se proceree el OCK An LOCKED), 
aie ACWelabielSeer eietevesgleeie, (Grier. A ledesh Ola delaiat ley)» tae 0) i) 
abn Mile @abibelee ch 36) 0) okie ke ba 
jeumuall ie Pe Oe FO) ONE id aed OA emi O |G aollaigil lan gelado) 2} oniyel) 


parent (ola lkio Gee), 8) Oa) ay 


parent diame @ er. 


return (parent); 


for (p = child; (p->p_treeflag & P TREE FIRST ORPHAN) == 0;) { 


/* Cannot use LIST_PREV(), since the list head is not known. */ 
[oo = OO SCM Sacene Goa 9) Ona olgenaon oNmeane sneiarehe, Veneer: 
(See ols eigha GE Malye) Uy, 
KASSERT ((p->p_treeflag & P TREE ORPHANED) != 0, 
Ma eWsteiale) Te Oia aii een es 1) ss 
; 
Sisleienue elena sigaicnc(one (We) Pes ona elelsighs len tergenign GS encUlers ‘onelerey 
(ob Koimolgvehavel call iet se abacione je 


return (parent); 





Now, the next step to see is how the kernel makes sys_getpid available to the userland. 


Our getpid syscall can be found in sys/kern/syscalls.master file 


20 is the syscall identifier, AUE_GETPID is the auditing event. Auditing in FreeBSD is the possibil- 
ity to log different type of informations from those syscalls: authentication, file descriptor opera- 
tions (via fentl call), userid/groupid changes. In our case, in fact, AUE GETPID is equal to 
AUE NULL, which means no auditing. At last, STD means it is a standard syscall. 


FreeBSD allows you to add new syscalls similarly to the ones above. 


For the example, let's make a syscall that returns the full command of a given pid (and if the pid is 
-1 then it would be the current one), which would have this signature: 


It might be advised to create a new file instead of updating sys generic.c, let's create 
sys _custom.c inside sys/kern folder 
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#include <sys/sysctl.h> 


#include <sys/sysproto.h> 


// Here this is just for the example, having a “optin” to enable our custom function 


Swen eMes age evel ollie vets qi) =? (0, 
Sods OL GIN i exe l—) ol bie yak CINDY NUM Fay —eyal ode elbhsye(ejiy, Oat) dl VE B sauiy 


Seeloike seein We iieeledkis Tela ei) 


am gue 
SDS CU VSITEM( GUCNalen Coleiavilere Aciguiaierelel suqulele Hoicna leis eenUls imei aa ehgver pougers 
{ 

Storie te OeoCe sor 


AGU R= Wala On een Oe 


Ol wie) amere ves Ih NO, —P ible 


memset (uap->buf, 0, uap->buflen); 


aide <i Caiakel oor We) urepere)i ee 0 a 
// We set errno if the buffer is too small or 
if (uap->buflen < MAXCOMLEN) { 
JOO nee terenenige MON eye 
error = EINVAL; 


OfONEte) oNbhes 
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// if we do not find our specific process 
Ciel O=2 > obcie = ary) ae 
if ((p = pfind(uap->pid)) == NULL) 
eG lee aerol ge Se weetibs WN) Se are 
error = ESRCH; 


GOEO.OWE, 


; 


PROC UNLOCK (p) ; 


1 Soe Bele ciehs ecole, 


/! Before copying to the buffer the command, we lock the process 


PROC LOCK (p) ; 


SH Wed elev Ace ya d ONUI eo aeeie)e Cerone Ly SM) A 


PROC UNLOCK (p) ; 


lq SUCNO Weigle a (anaus@ue) 





To declare our new function, we need to add this in sys/kern/syscalls.master file: 
Note: the identifier must be unique, we just increment from the previous. 
Then, in our FreeBSD source folder, we could type this: 


Now there are new entries in sys/sys/sysproto.h header, first we have our userland struct 
arg: 


and our kernel function declared: 


We need to declare our new file sys_custom.c to the sys/conf/files file as below: 
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So the next time we compile the kernel it will be taken in account. Last, we need to make avail- 
able this function exporting its symbols: 


Once we have updated our system, our new function and our new sysct1 ought to be available. 


We could test this new function with this short sample code. 
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If you think a new syscall is needed then you can either discuss it in the dev freebsd mailing list or 
proposing a patch. Maintaining locally this kind of code change might bring difficulty when updat- 
ing the source with new official syscalls being added and conflicting identifiers. At least, hopefully 
this article gave you the taste to dig into this topic. 
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FreeBSD Kernel 


In this article, we will give an overview of the nature of the 
FreeBSD’s kernel. The important configuration files will be 
explained in addition to learning how to compile the whole 
system with more options, with more debugging informa- 
tion. Very useful for kernel development. 





What do you need: 


« FreeBSD 10.x. 


- Machine with at least 4 cores is recommended for the system compilation. 


¢ Genuine hardware or virtualized environment as your convenience. 





1. The FreeBSD kernel 


FreeBSD, like many kernels, is a monolithic kernel with loadable module support. Hence, it is pos- 
sible to build FreeBSD’s kernel with all needed modules statically or, those ones that support it, 
as separated dynamic loadable modules. 


The latter ones can be loaded and unloaded at will at boot time (<name of kernel mod- 
ule> load=“YES” in /boot/loader.conf file) or via kldload/kldunload. 


To have an overview of all currently loaded modules, you can type kldstat. 


For example, the output looks like the following: 


Id Refs Address Size Name 


IME c 0h Grp e rene cree mon O40) OOOO Secon tons doperatou 
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If we add dtraceall load=“YES”, we will be able to use Dtrace framework facility. You can 
find an excellent introduction of dtrace in the BSDmag issue of December 2014: 


Indeed, Dtrace can be very useful for tracing syscalls. 


In order to build the system, we need the whole source code, hence the kernel and the userland. 
The userland is simply all the base utilities of FreeBSD. Both kernel and userland code are consis- 
tent and tied together, available in the same subversion repository. Apart from pure BSD codes, 
we can find GNU libraries and software (called contrib code). In addition, for ZFS, CTF (Compact 
C Type Format debug section, similar to DWARF format but reduced in term of size) and DTrace 
proper compilations, some CDDL codes are present. Happily, FreeBSD is provided with subver- 
sion in base, suffixed distinctly to avoid colliding with the port version. 


Check out the source in /usr/src via svnlite 


svnlite co ht | /usr/src (or you can check out the 
current branch with much newer sece but with more instability, you can just replace stable/10 by 
hed). 
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SFreeBSDS 
On the handling of kernel options 


All kernel options should be listed in NOTES, with suitable 
descriptions. Negative options (options that make some code not 


compile) should be commented out; LINT (generated from NOTES) 
should 


jaereresi(| abd ecw yoltsuan (bel smelesel—\ a chome oles rs Lele oye dha cunblelmcie 1e/ulouchblia-r(e) oe il Ola mei Ursi ih ale! 
code so that a single option only switch code on, or only switch 
code off, to make it possible to have a full compile-test. If 
NeCSssal vy wVOuUscam echecko tore COMPILING MEINE srotcer maxamun code 


COW SICEIO'S - 


All new options shall also be listed in either "conf/options" or 
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this 1s a kernel-wide option (used just about everywhere), or in 
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Note that the effect of listing only an option without a 
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header-file-name in conf/options (and cousins) is that the last 


convention is followed. 


This handling scheme is not yet fully implemented. 


mem ai(vohes ene on edeliic neal. 


Option name filename 


If filename is missing, the default is 
S] SE IE Geel ah O se ©) 9) WAL ONG) Mae HONS lam Oreos nf) 

INC DisisiUE ere1w | reieie sia) 

IVNGIoU LB) Dials (oles Veleieneeiniel 10 

AHC ALLOW MEMIO opt _aic7xxx.h 

AHC _TMODE ENABLE opt _aic7xxx.h 

AHC DUMP EEPROM opt _aic7xxx.h 

(Mal@e IHS ICE selene gsbien1>-o..o5 2 

Ns Ge DIN a UG Ole MbS we) Oe se Inber y.o.0.4e 4) 


AHC_ REG PRETTY PRINT opt aic7xxx.h 


FN eID BID eOlE ste) hey iker/ech.o.c ni al 


MnO DIES 1 3G of aus ae eee rele jah o.4a) 0) 
AHD TMODE ENABLE opt _aic79xx.h 


AHD REG PRETTY PRINT opt aic79xx.h 





¢ For each line, the option’s name then the file created with the relevant preprocessor defined. If 
the option is present in your kernel configuration file, let’s say AHD_DEBUG OPTS, it is possi- 
ble to test if AHD_ DEBUG OPTS is defined and providing some contextual code for this option. 


¢ Let’s imagine we did a new shiny kernel module, we could add our proper line in this file. 
BSDMAG opt bsdmag.h 


¢ Another important file is /usr/src/sys/conf/file. 


BSD 


MAGAZINE 


46 


BSD 


ANIZVOVW 


47 


¢ For each line the relative path to sys, the type of module, if optional it will be compiled with the 
(lower case) option name written afterwards. 


¢ Again, with our new module, we can add in this file our specific kernel module C file, let’s say 
workshop modulel. 


So now we can create a custom kernel config. Let’s call it WORKSHOP. 


(it will pick up the new WORKSHOP configuration file, by default it is the GENERIC one) 
Steps to build a system: 


First, the userland needs to be compiled. 


If your machine has multiple cores, it is advised to use them for the system compilation. 


It might take several hours depending on your current configuration. 


Possibility to do make -j<number of cores+1> buildworld kernel (ie create and install 
the kernel at once). Restart in single user: 
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Eventually mergemaster -p then make installworld: 


mergemaster -FUi => mergemaster will try to merge various configurations files and asking 
you how you wish to proceed, merging as possible, replacing with a newer one or keeping the ex- 
isting. 


Restart in normal mode. 


You should have now a workable system with the latest fixes/patches for the 10.x branch. But, as 
a developer, we might need more info from the system for debugging, studying the core dump af- 
ter a system crash/kernel panic. It is advised, as kernel developer, to enable kernel core dump 
writing (could be enabled when you installed FreeBSD or, afterwards, can be enabled via dumpdir 
rc.conf variable) at the cost of disk space consuming (can be potentially important, deleting old 
ones is necessart). They are, by default, located in /var/crash. In order to debug a kernel crash 
dump, the kernel compiled with debugging symbols, kernel.debug, is necessary. gdb can already 
be used this way. 
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Reading symbols from /boot/kernel/dtraceall.ko.symbols...done. 
Loaded symbols for /boot/kernel/dtraceall.ko.symbols 

Reading symbols from /boot/kernel/opensolaris.ko.symbols...done. 
Loaded symbols for /boot/kernel/opensolaris.ko.symbols 
Reading symbols from /boot/kernel/dtrace.ko.symbols...done. 
Loaded symbols for /boot/kernel/dtrace.ko.symbols 

Reading symbols from /boot/kernel/dtmalloc.ko.symbols...done. 
Loaded symbols for /boot/kernel/dtmalloc.ko.symbols 

Reading symbols from /boot/kernel/dtnfscl.ko.symbols...done. 
Loaded symbols for /boot/kernel/dtnfscl.ko.symbols 

Reading symbols from /boot/kernel/fbt.ko.symbols...done. 
Loaded symbols for /boot/kernel/fbt.ko.symbols 

Reading symbols from /boot/kernel/fasttrap.ko.symbols...done. 
Loaded symbols for /boot/kernel/fasttrap.ko.symbols 

Reading symbols from /boot/kernel/lockstat.ko.symbols...done. 
Loaded symbols for /boot/kernel/lockstat.ko.symbols 

Reading symbols from /boot/kernel/sdt.ko.symbols...done. 
Loaded symbols for /boot/kernel/sdt.ko.symbols 

Reading symbols from /boot/kernel/systrace.ko.symbols...done. 
Loaded symbols for /boot/kernel/systrace.ko.symbols 


Reading symbols from 


/boot/kernel/systrace freebsd32.ko.symbols...done. 


Loaded symbols for /boot/kernel/systrace freebsd32.ko.symbols 


Reading symbols from /boot/kernel/profile.ko.symbols...done. 





Like the userland gdb’ s counterpart, we can use backtrace (bt). 
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257 #ifdef KTRACE 
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Then, for example, going up in the stack frames calls and so on... 
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For more information about gdb, a good helpful workshop exists about this topic: 


Indeed, especially if you run the -CURRENT branch, the kernel can crash for various reasons and 
this gdb like tool is handy to have a basic understanding of the reasons... 


Indeed, FreeBSD does not rely on the Giant Lock model anymore, it is based on fine grained 
level process locking/unlocking. Hence the resulting programming can be tricky and it is easy to 
get lock contentions. 


With this workshop module, you learned the basics of kernel custom configuration, compiling the 
whole system. 


kgdb 

GNU gdb 6.1.1 [FreeBSD] 

Copyright 2004 Free Software Foundation, Inc. 

GDB is free software, covered by the GNU General Public License, and you are 
welcome to change it and/or distribute copies of it under certain conditions. 


Type "show copying" to see the conditions. 
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There is absolutely no warranty for GDB. Type "show warranty" for details. 


This GDB was configured as "amd64-marcel-freebsa"... 
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NetBSD and pkgsrc-wip* 


For this mid-summer, we will approach a lighter subject, 
NetBSD and its ports system. Pkgsrc is the framework to 
build third party packages for this system. We will see how 
to create a package and hopefully submit it. Hence, the 
pkgsrc is supposedly already in your system. Otherwise, a 
full quide is available here. 


1. Environment 


It is recommended to install pkglint which will serve to produce a better package. Indeed, as its 
suffix suggests (lint, the historical C code analyser), it will check the whole package structure, the 
Makefile, the checksum and so on. 


Secondly, you need to choose a main category for your library or application, even if your future 
package can possibly recover several. For the article, we will choose security/yara, the popular 
malware searcher library, as an example. 


2. Makefile 
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NetBSD 


WRKSRC= S{WRKDIR}/yara-$S{YAVER} => WRKDIR represents where the source port 
will be extracted (generally it is work/<package name>-<version>) 


USE TOOLS+t=pkg-config automake autoreconf => Necessary tools to build the 
package. Could be cmake, perl. They will be installed if not present 


USE LIBTOOL= yes 
GNU_CONFIGURE= yes => Uses GNU version of configure script 


PKGCONFIG OVERRIDE+= libyara/yara.pc.in 


pre-configure: 


cd S{WRKSRC} && autoreconf -fiv => Wecan override many sub tasks, related 
to different steps, before, after the archive extraction, configure, build, installation and so on 


-include "../../security/yara/Makefile.common" => Makefile.common is used 


by at least two packages (in our case py-yara) and it regroups common information, could be 
the dependencies, the version ... 


.include "../../mk/bsd.pkg.mk" => Mandatory file to include, it contains the main 
necessary variables 


Now, let's have a look at the Makefile.common 


# sSNEtBSD: Makefile.common,v 1.3 2015/06/14 21:28:44 pettai Exp S$ 
# 
# used by security/yara/Makefile 


# used by security/py-yara/Makefile 





We talked earlier about the DESCR file, it is simply a text file which describes more completely 
the package in question like below. 


We also need to know the list of files to be (un)installed relative to the variable PREFIX (usually 
/usr/pkg). It is the role of the PLIST file. 
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bin/yara 

bin/yarac 

include/yara.h 
inelude/vyara/anocorasick, ht 
include/yara/arena.h 
include/yara/atoms.h 
include/yara/compiler.h 
include/yara/error.h 
include/yara/exec.h 
include/yara/filemap.h 


include/yara/hash.h 


include/yara/libyara.h 


include/yara/limits.h 
include/yara/modules.h 
mnielude/vyara/oojyect. nh 
include/yara/re.h 
include/yara/rules.h 
include/yara/scan.h 
include/yara/sizedstr.h 
sin gloubKe ley Ayzeliately foie raises i hcuea) 
include/yara/types.h 
include/yara/utils.h 


ilabley lation zenerceyd be! 





Sometimes, the software in question needs to be patched in order to work properly. The patches 
subfolder should contain the necessary diff files, by convention named patch-<path to the file, 
dashes replaces by underscores>. In our case, we have patch-libyara_ proc.c which just 
needs to add NetBSD support. The patchset is created via make patches. 
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Eventually, if it’s a library we can create the buildlink3.mk file, if another package needs yara 
library as a dependency, this package just needs to include this file: 


Once we have all the pieces needed, we can finally create our distinfo file which stores the check- 
sums of the DISTFILES and eventually the patches. It is created, ideally, via make makesum. 
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Size (yara/v3.3.0.tar.gz) = 7634474 bytes 


SN RVIL, “iGerelere!lay sl aLlenccligtey,  SMaKoke ney = 
erode Oy OMboKey OKC Mexers Tees Tone eum e\cyorc1 lol Otel OM cusae lore: 





7. Checking the package 


pkglint will display every part of the package which is not correct, the FATAL messages must 
be taken into account, some WARNING messages, too. 


pa bee platigye 


looks fine. => Ideal, but a correct package can have few harmless warnings too. 





8. Submit 


There is a project which aims to get more people involved in investing their time to create pack- 
ages for pkgsrc. It is called pkgsrc-wip and can be found here hito://okgsrc-wip.sourceforge.net 
and if your package is correct enough you can get commiter bit. | hope this article gave you the 
taste to create yours. 
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HardenedBSD, always ahead in security 


Previously, | focused on the HardenedBSD project, handled 
by Oliver Pinter and Shawn Webb, especially the Address 
Space Randomization Layout feature. The HardenedBSD, 
also has other features available and I'll try to describe all 
the features. 


1. arc4Random and Chacha 20 


Currently, FreeBSD uses the RC4 stream cipher for the arc4random family functions, both in ker- 
nel and userland side. These functions serve many purposes, for example, in the kernel side; it 
allows the creation of proper randomized processes id, the stack protection canaries, and the 
HardenedBSD Address Space Randomization Layout uses it as well. 


In the userland side; openssh uses it widely and is also used in the stack protection counterpart. 
It is generally an important piece of software. 


Recently in the last Hackfest (and previously in the last EuroBSDCon), Theo de Raadt discussed 
the arc4random OpenBSD’s version and raised the need to move on from RC4 to a stronger 
stream cipher. Hence, the invention of Chacha 20, implemented after the 5.5 release. 


Subsequently, we decided to update the HardenedBSD as well, in both kernel and userland side. 
In the kernel side, the challenge was to keep it SMP safe while keeping the code change smooth 
and wise while in the userland side, the challenge was to update the fork detection. Indeed when 
a fork is created, the reseeding is triggered. Usually, getpid function is used for this purpose but 
we thought there might be a better and more solid approach. M. Dempsey, an OpenBSD contribu- 
tor, provided a new minherit flag, MAP_ INHERIT ZERO to ensure that the memory map is prop- 
erly zeroed in this case. So, for HardenedBSD, anew INHERIT ZERO flag was added. 


Related to this, a new system called, getentropy was added as well. Basically, it fills a buffer of 
randomized bytes with maximum of 256 bytes. It serves more as an initial input for randomization 
rather than using it directly. Hence, for example, it can replace a couple of sysct1/KERN *RND 
Calls. 
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Again, we got inspired by OpenBSD and added some of their useful libc functions. 


getdtablecount gives the number of file descriptors per process. It can be helpful alongside 
getdtablesize. 
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#define FDRESERVE 5 


Ae gS 
Mada) (Gite kOe, 6 Clic ras” aa Cn. al)) 


{ 


if (getdtablesize() - getdtablecount() < FDRESERVE) 


Snaiay quibe. hsql bal amhiakes (a 6)e wank agahiben ol ysleagite) etenas 9 ye 


reallocarray checks some potential overflows (but does not zerofy) 


ais aven hekelss ees aelliklenedqe: 


diane elas —<angue lab 


ahah e 
iivctltal Gigheuwcwderer ielars ys relate pis Ih) 
{ 
Team e eeO a care Fe 
// 1.€ same as realloc(NULL, 2 * sizeof(*p)); 


JOS Aa eeu elelc mata ly AGN IUMENM yy ae ead Vong od 





A slightly different version of stricpy is provided. stricpy usually guarantees a zero at the end of 
the buffer. But the buffer does not sanitize the potential remaining bytes. So our version combines 
both stricpy and strncpy advantages at the cost of a slight performance hit, only HardenedBSD, 
at the moment, does it. 
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For the last, the crypt API was updated recently. Two new functions were added, 
crypt newhash and crypt _checkpass. The latter provides an easy interface to test the valid- 
ity of a password, while the first allows the creation of a hashed password. Once again, inspired 
by OpenBSD. 
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A secure webserver on FreeBSD with 
Hiawatha 


In most cases, when it comes to choosing a web server, 
Nginx comes quickly to mind (I personally appreciate this 
one a lot, no doubt about this). However, an interesting alter- 
native exists that embeds some very nice features, an alter- 
native called Hiawatha. 


1. Features 


What makes Hiawatha special? First, its code is well audited and famous for its solidness in term 
of security. Apart from having CGI/FastCGI support (hence possibly making dynamic website with 
PHP-fpm), SSL, lpv6, Virtual hosts. it also provides a protection against SQL injection and XSS, 
CSRF natively. It might lack third party modules support, as the architecture does not allow it but. 
Hiawatha has Reverse Proxy support! 


Luckily, FreeBSD already has a package / port. So once installed. 
2. Configuration 


Let's configure it. Here’s a sample configuration, a FastCGI's one. You can see that it appears 
very human readable. 


ErrorHandler A0A = / 404 hen 


lswingtclabave pe | 


Port = 80 
Interface = ::l 


MaxKeepAlive = 30 
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# Two first secure measures, limiting the client request size and the max time for 


# the maximum time for a client's connection kept opened 
MaxRequestsSize = 512 


TimeForRequest = 3,20 


Isuliave akiavey 
Port = 443 


Interface = 


SSLceertfile = hiawata.pem 
ReguireTLS = yes 
# Add X-Random header with a 256 value long 


RandomHeader = 256 


# A feature to make decisions based on url regexes 
Url roc lichen, 

ToolkitID = phprewrite 

Method GET 

Match /private DenyAccess 


Match “/page/(.*) Rewrite /index.php?page=$1 





Security 


FastCGliServer { 
PastCGlive — “PHP 
UseToolkit = phprewrite 
# Can be the path to the unix socket too 
ConnectTo eek, OR UR aie eal Onis 


Extension jOlayo 


Wolieeibyeuliskerst eee), 


EOS cicils —= WWW.example.com 





WebsiteRoot = /usr/local/www/hiawatha 
AccessLogfile = /var/log/hiowatha/example-access.log 


ErrorLogfile = /var/log/hiowatha/example-error.log 


# Althought those directives can provide protection, they are of course 


# not 100% reliable 
(Eigie Vicime\erolnde Vers 
Pigs Wroldie SONRIL ves 
PREVEnNEXS5:— Ves 
# A body which matches this regex is forbidden 
DSN Astle a tO eee sO Slenc olen, Wao Oise silence) ca ws 
Use Seis cil Nes) 
TimeForCGI Ss 


# By default it is index.html 





Security 


StartFile = index.php 


So, for the moment, nothing special and never seen before. Let's dig in more ... Indeed, with Hia- 
watha, we can set some banning policies like below: 

# If in our VirtualHost above a user had the bad body content 

BanOnDeniedBody = 300 

# If too much malformed HTTP requests are made by a client, it’s banned for 1 min. 

BanOnGarbage = 60 

# Banned if the request size exceeds this size 

BanOnMaxRegsize = 512 


# Anti flood measure, here if the client does more than 20 requests per second, it’s banned for 
mallee 


BadOnFlooding = 20/1:60 

BanOnSQLI = 60 

KickonBan = yes 

# Reset the ban times if the client attempts to connect when it is already banned 
RebanDuringBan = yes 

GarbageLogFile /var/log/hiawatha/garbage.log simple 


Also Hiawatha provides a separated feature, called Monitor, a feature a bit “a la” Munin but well 
adapted to this server. 


# This IP address will be allowed to download the event log files for its analytics 


Monitorserver =.197. 6s 1. 2 








As mentioned before, Hiawatha does not support third party modules, so if you wish for your serv- 
ice to make a decision based on country per IP with geoip, for example, you can use Hiawatha as 
a Reverse Proxy in front of Nginx. It’s as simply as it is shown below inside your VirtualHost set- 
ting. 


Wiemiaeibicllskessice | 
Hostname = Www.myreverseproxy.com 
[igie Vicimelersinds 9 gees! 
PmevVeniies © lal ves 
Ie ile WASEDA SyS. =) aVKers) 


ReverseProxy .* http://192.168.1.2:8111/geoip 


# Cache internally those contents from reverse proxy requests per extension 


CacheRProxyExtensions = gif,png,j]peg,css 


CacheSize = 256 





3. Conclusion 


Hopefully, this article will give you the curiosity to consider this approach. For what it provides, it 
is impressive regarding the fact that it is a one man work done since 2002 (by Hugo Leisink). 
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NodeJS and FreeBSD - Part 1 


Nodejs is well known to allow building server applications in 
full JavaScript. 


In this article, we’ll see how to build nodejs from source 
code on FreeBSD. You will need autoconf tools, GNU make, 
Python, linprocfs enabled and libexecinfo installed. GCC/ 
G++ compiler suite (C++11 compliant, ideally 4.8 series or 
above) or possibly clang can be used to compile the whole 
source. 


To start, we need the nodejs source code from this url hitp:/,www.nodejs.org/disi/atest where we 
can find this archive (during the article writing, the last version known is 0.12.2), 
node-v<version>.tar.gz. 


Be prepared to be patient, you have enough time for a cup of coffee, the compilation time needed 
can be quite long... 


Once downloaded and extracted, the famous command trio needs to be typed: 

« ./configure --dest-os=freebsd 

* gmake 

e gmake install 

It’s pretty straightforward on first glance. On FreeBSD, when v8 is compiled we get some compila- 


tion errors: 


(etleW ANG bias Fy DNAs) SW SMEs aN 1@ ihe. Goy 2 SacBee ID Mey Verena Wis ti gacy | WicsIDI NV AUe lich sli 
DLE ZAPPING' -I../deps/v8 -pthread —-Wall -Wextra —Wno-unused- 
parameter -m64 -fno-strict-aliasing -I/usr/local/include -03 


~EtUnGelOn—seChlOnsme at OAtaaseCEloOnsS so Eno-OMdte—- ii amMe— po imGer wo Edd tas 
SSCLiIOns --LeumelLIoOn=secLione =o. —fio=t bias —iie -excepri onc. —-MMpe-MpE 
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/root/node-v0.12.2/out/Release/.deps//root/node-v0.12.2/out/Release/o 
bj.target/v8 libbase/deps/v8/src/base/platform/platform-freebsd.o.d.r 
elie “aon Eo) 

/root/node-v0.12.2/out/Release/obj.target/v8 libbase/deps/v8/src/base 
/platform/platform—freebsd.o 

Hayy lal olsy acy sualey Mevehciey, jellycy eh mola iy odbctrda oud giiiels(olsiolyiote! 


-./Geps/73/src/base/ plattorm/plattorm—treeosd. cer 15o:11-verrers mem— 


ber reference base type 'int' is not a structure or union 


rafepen bios e]bEsiok Mersie|. oi Gelatela <folllaklenats aanis.Velelial yoyo) (eucrslia)b, NOR NerseNals leibye yal oar 


Pe, 060s / 787 Sec/ basce/ plat form, plattorm kresposd. ce: lol: S32 errors use 
ae anslialel ie lleva eli ale —iqealenk— ie Ui Mee ING) adensieneet day 


MAP PRIVATE | MAP ANON | MAP NORESERVE, 


ran 


../deps/v8/src/base/platform/platform-freebsd.cc:263:48: error: use 
ONE youalel Selo heel able agicaleeene Ven INO I sUchsnchi sa fla ¢ 


MAP PRIVATE | MAP ANON | MAP NORESERVE, 


w~ 


(o0/ JE@S/ 7S/Sre/base/platformy plattorm—freebhsd. ce:20 le 40s werror:suse 
ORE wibleleloie beta <(el sto l—Vaneabieatoae: A {UNE UNO I Un sudie ys tay 


eee led IEMA Sa) Ee TAINIGING s | ici NOI S IR Acy ])) ice sal cD 


wn 


4 errors generated. 


Ok, so a result variable ought to be a std::vector but it’s considered wrongly as an int and further- 
more a wrong mmap flag is used. Let’s fix it! 
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std: :vector<SharedLibraryAddress> result; 


SIBCUE de Seroiikone suane, Nee. ING Min = ale earl 


int fd = open("/proc/self/maps", O RDONLY) ; 


iter Chee 0) eee eile Ces tMlaize 
while (true) { 
(@lgisvel yoiele ne Joie vere Syan| EAL 
eV (ola leybnene San (0h 
clolohia: sibiee req] 
acids Dat eer EO 
ale Gel ers b Mie St aSelel(amel. Seleleha Lewhaimcuer a)" ear) ie 
hue) Waekepsn Olle eK oye te agersl.an 
UiSLOQned Stati —mor ring lonong add oun ier)y, 
reper OUlger ty ere —velel, (nmol xcNe olay Jelbnmim um van Hy he 
ste Galore b Mine ee yh Nee c7sh.qr 
BIE Geto lole: Jeecrumsrallyal| © fax We) lenaicret <r 
(SHG es Sctsvetel(eiel. clelolie elbeisie ae Wa, wedi 
ie Tessie Ke tod hone syed <r 
Dhaheyibe igheelracnqlel 2 TSuvneubiguesll elbyepale(beve lobe le) b hee Suny pe 
(elateta Fle) beim — ag PUreWog ENE Al sti 
PA eY NORE ietsw eeveiel Po Ie 
do { 
ION A ote te ree hci at 


alee, Meni ieisien aa srelel SF UV Nae AN (Saisie ih) 





NodeJS 


break; 


ig cowo le, engi cioleMnel. SeWima cia ae Ney amicton qcveleys iL y 


Apparently, there are two different variables with the same name. Let’s rename the second, the 
int type, to res, for example so the vector result variable can legitimately call push_back method. 
That fixes the first error. 


std: :vector<SharedLibraryAddress> result; 


SHB cuesne: clefolghsie, sadn, INV, MENG Mbt cs Ale eAee 


int £d = open("/proc/self/maps", O RDONLY) ; 


sm, GmGl eS ON) Getsleibmew, aacio edie, 
while (true) { 
lacie ccholole: deypveie eq) Ab AL i 
acd valour he 2) 
elolole ceibhmie sre) 1n) 
ele lola lel pinme eva (AO a bir 
Me eisie diershol(MimelM Velolela Vel umin cna re Pare fis) 
if (res < 8) break; 
(Dbalisiiie pavers) Folwtshigw, —2 ssuvsedligiosWepbiojaye Ausko lola, Jelbhes ie) iy 
(aston =P lidlere en ieiele vole lola sow ummsna. ae Var ye 
if (res < 1) break; 
he, Westelehalenbomitcrali ad) oi AINE Yona craic 
reforoy Ga? Viet aiclel ale ly actclolnay Aolb pene Suet ae 4s ey 


if (res < 8) break; 
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Dhotenbe felevol Meagtcl, = (Siena kiohonlbonkyeialencetoloba Velbiaid=ye)) e 
elacwer Jee ie Ure Ne Aue a oe 
INGE TO Wioter tai vaiel SStem Ilr, 
cle" a 
TOR AG = ey melee laser 
ibe, MCR cetst be cyelsh STUN ee Mia evisk Mika) 
loueSreller 


de chey ee rel —rohol namely Gelb hash es Mare Noam icrseite leh aly ie 


Let’s have a look at the mmap problem. 


MAP NORESERVE is a specific flag which guarantees no swap space will be used for the map- 
ping. However, it is a flag usable on Linux and Solaris /Sunos. 
mmap (OS: :GetRandomMmapAddr (), 
Salas 
PROT NONE, 
MAP PRIVATE | MAP ANON | MAP NORESERVE, 
kMmapFd, 
kMmapFdOffset) ; 
= 
mmap (OS: :GetRandomMmapAddr (), 
Spa ay 
PROT NONE, 


MAP PRIVATE | MAP ANON, 
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kMmapFd, 
kMmapFdOffset) ; 
void* reservation = mmap(OS::GetRandomMmapAddr (), 
na Sle Svene awh 
PROT NONE, 
MAP PRIVATE | MAP ANON | MAP NORESERVE, 
kMmapFd, 


kMmapFdOffset) ; 


void* reservation = mmap(OS::GetRandomMmapAddr(), 
1a SHO SVS 1. Seialra sy 
PROT NONE, 
MAP PRIVATE | MAP ANON, 
kMmapFd, 


kMmapFdOffset) ; 


Once modified in every mmap call, we can now retry compiling. However, we get another compila- 
tion error. This time, it casts a pthread self returns call to an int. 


deps/vo/stc/ base, plat torn plattovm—-oocis ce. solar neruonr: 
SHereNs de! MeeVee. Ohua eit GOhelgiattels Ie) «ied ack Vychelenalstsvol Fou) piwles: Makigha se ils) Gieleye Joule 
lowed 


TNE ONCIAL Toller WedkOk Terhore salighes * (Oise ie. esleulhim () Ie 








The problem is, on FreeBSD, a pthread_t type is not an integral type at all but an opaque struct. 


Instead, we might replace this line by: 





IGS UNDNAINS Aoherecmlien etches ecole sal mechliehecnaoneour ferhs nce idl suege nese Sena evel al sorodlne A yh) 


Now we are finally able to compile. After a couple of minutes, it is finished but we have still one 
source to update: 1ib/dns.js. Add these two lines after line 127: 


if (process.platform === 'freebsd' && family !== 6) 


hints &= ~exports.V4MAPPED; 





Because FreeBSD does not support this flag, it ought to be cleared. 


This is all for compilation and it is ready to be used. Next time, we'll have an overlook in the appli- 
cation’s building part and ought to see the potential of this library. 
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NodeJS and FreeBSD - Part 2 


Previously, we've seen how to build NodeJS from the 
sources in FreeBSD with minor source code changes. This 
time, we'll have an overview of the application’s build proc- 
ess. 


Numerous excellent tutorials exist to build a nodejs' applica- 
tion in pure Javascript. However, the possibility also exists 
to build an application natively in C/C++. It is exactly what 
we're going to see. 


1. NodeJs application structure 


We only focus on the modern way to build a native application. Before, we had to do a node-waf 
package via a Python script. It is deprecated and was replaced by node-gyp. This is a basic gyp 
project structure: 


A binding.gyp file describes the source code to compile, the package name, eventually the neces- 
sary compilation/linker flags. Let's start with the usual Hello world example, quite FreeBSD. 


2. Hello world 


First, we need an entry point, an initializer, from which we will export our functions to nodejs. 
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And to register our module. 


NODE MODULE (freebsdmod, Init) => Note that there is no need of a comma after this 
macro 


Very well, but for the moment our module is not useful yet, we would need at least one feature. 


Let's imagine a simple random function that uses, internally, one of our arc4random family func- 
tions, a function that will be called from a nodejs script. The signature of this function would be: 


eo Mecha loMiUO slate, avmeuna Oi nlGKe eamonglerc libero! <lhiamme.arncus wNyedapiee. 20) i 


We can imagine that, from the nodejs script, we would like to provide a max value limit as a 
unique argument: 
#include <stdlib.h> 


tinclude <node -h> => includes. both node and v8 structures 


using namespace v8; 


voldUuRandom (const yhuncelongallbackinto<Valic= teargs) 4 


Isolate *isolate =Isolate::GetCurrent(); => Here, we get the current v8 engine in- 
stance 


unsigned long value = 0; 
sine wae piere Koay —yaye meds (meh) 
isolate->ThrowException (Exception: :TypeError ( 
String: :NewFromUtf8 (isolate, "Needs an argument"))); 


if (args[0]->IsNumber()) => the arguments are conveniently wrapped, we 
have access to the caller arguments. 
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WcU bin —F Foie esiok fe ctsye “bhaiswoia( aol MMe qiep al chee (e ne chg (elo apigmlem ona iM eleeFsoii0) |i 
>NumberValue()); 


else 
isolate->ThrowException (Exception: :TypeError ( 


String: :NewFromUtf£8 (isolate, "The argument is not a 
number"))); 


args.GetReturnValue().Set (Number: :New(isolate, value) ); 
i 
yiconiiel aiieiie (Uskehavelii= a@lomicvene> t=) 4e\oneiers))) 


{ 


NG )EIA SUMAN MM ODN Gy elena ior. Aaacn tole fe Ustenaslop hrs eon Noe mma pleulh lye wey qeroneis 
our Random function here 


Now, let's have a look a the binding. gyp file. 


"targets": [ 
at 


Mieretaele (cies AplsiieN Was iciclersteiilolelys 4 mciona tei oigeis: MEldksh Agel “one, Genblin pitlelole 


URSLONbNaOlNcwye. |) erase cyoluerekaerenn | 
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Simply, as it is, it is sufficient for this first example. Now, we can compile our module. 


> node-gyp configure 


Ex s0(Oj6=ret OVS aS) wail! 


We can now test with a simple nodejs script. 


var fmod = require('./build/Release/freebsdmod') ; 
WiSi@ a MIOGh Nd (Ole lmnarchaverovaianmgepae ete ALO bays A ac 


console.log(rnd); => Should print a Significant numerical value 


3. Wrapped objects 


Apart from making atomic C++ functions to export, we also have the possibility to handle more 
complex cases, by making wrapped node objects. For this example, let's use yara library, the mal- 
ware's tool. The binding.gyp file would look like this: 


soepewarsfeqeree 4” Ml 
{ 
Shercliagefone: Werelicnute wmaigewerey gieiol= Ar 
UesKopliaiorctstine |i reclmshavorel=uaielo.s ill 
jalboven Gbtel=\ Ne bigot eal Gbics a) ale fer Wlysncsteleblol=s ia. 


Milsienactevketowe re ale iy (brote/Mileyariy (ibibo wes) i scliers || 
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A wrapped object must inherit ObjectWrap class. 


#ifndef YARANODE H 


#define YARANODE H 
#include <yara.h> 


Lae divlol=s Kagholeksen ie 


#include <node object wrap.h> 


SNe sesh: Bicoahel perelchabukeveleaiahenee verohetcie Selstsae “ae hehe, “eloiicim velsicne oA Ayienleloun is 


class YaraNode : public node::ObjectWrap { 
private: 

BS O(6) NBA CHS ean dear, 

nipeWee Waianae BU tei 

explicit YaraNode(); 


~YaraNode(); 


Statrcyvord New (const. Vo ~sbuner ronCallbackinto<v7ss.-Valie>s); 


static v8::Persistent<v8::Function> constructor; => Contrary to the Lo- 
cal handles, a Persistent storage is independant to any HandleScope, valid until cleared 


SHsiGaken \eonlol wVeloleus a uouleropaicie waerm mab uguelmanang Ochi Libeyelel cll gnmenqitem maleeM blame soe a 
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Sue sieaker aigouuel (Sieish als IMs Melolasie sie mie ola lene amon sallme siol ciiqumer aids. Vicklbise ae) 
joibheplinkie:+ 
SUE heaien afejalye P clitelakios Gigorenalal yale llores l@le gi ssene 4) 


SUEvswemce? cael a Agee cwesi is) 4 


The Persistent storage will serve us for the YaraNode initialization from within the Nodejs entry 
point: 


#include "“yaranode.h" 


using namespace v8; 


NASLIMS Kole lhanDMNetolenemame Keucixee, eoleusie, Kelaksiay way “ameye, line arey 
Constr Chary message, sVOld | “porivatle) ar, 
Isolate *isolate = Isolate::GetCurrent (); 


if (message) 


isolate->ThrowException (Exception: :TypeError (String: :NewFromUt 


isolate, message))); 


} 


Perstetent Funct rons Laranode: const ructon: 


YaraNode::YaraNode() { 


Wag SUecMe ONS! -—liae al gulqanivellial ass) (ur 
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aigie: Gerster neots! ——) an Olay TSH UICIG ISIS) = a 
ie Tevomonlil sie Tene Svouuien Sarde.) 


Se Seto msubik ong “crower ereli cielo <i gers. selelelcb alee INIUNn Ih) 


YaraNode::~YaraNode() { 
if (yrstatus == ERROR SUCCESS) i 
Nien cele eel Sa el verona cs) fy 


Aen ety aroha St) 


void YaraNode::New(const FunctionCallbackInfo<Value> &args) { 
Isolate *isolate; 
ielers Wi ibinlele none wie none 


isolate = Isolate::GetCurrent(); 


HandleScope scope(isolate) ; =>AHandleScope is responsible for all following 
local handles allocations 


if (args.IsConstructCall()) { => var yr = new YaraNode(); 
YaraNode *ynode = new YaraNode(); 


te /ioOde. estate) Shon sUeeitoe) 
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isolate->ThrowException (Exception: :TypeError ( 


String: :NewFromUtf£8 (isolate, “yara could not be in- 
stantiated"))); 


ynode->Wrap (args.This()); => Here we wrap our YaraNode and can be unwrap as 
will as we Il see slighty later 


args.GetReturnValue().Set(args.This()); => We return basically the 
wrapped yaranode object to the javascript caller 


} else { => YaraNode called as classic function 


eictone L lkjse sule<imiustejeatee arnt austeneuerey. (elelats erasteuctene) (pete. \ {Vee u ie 
here our persistent storage to instantiate our YaraNode instance 


args.GetReturnValue() .Set (ctor->NewInstance()); 


void YaraNode: :AddRule(const FunctionCallbackInfo<Value> é&args) { 
Isolate *isolate; 


shane aaer c— 10NA 


isolate = Isolate::GetCurrent(); 
HandleScope scope (isolate) ; 


YaraNode *ynode = ObjectWrap: :Unwrap<YaraNode>(args.Holder()); => 
Here we unwrap to access a YaraNode object field 


Ae, OS iT iteia, (ilar oe mal ue 


si AHOS Bye Sey 





NodeJS 


for (i = 0; i < args.Length(); i ++) { => addRule method, from nodejs 
script, is called like this addRule(<rule1>.,...,<ruleN>); 


Him, (Cs hare ce Wall poe alecvsnewaslagvad yh = 
eroleksies veletcne | wnaba kon 
Serres Ls Va biiewiieu c(t Ss | I= loo ur Bing. (ae, 
rule = *rrstr; 
ia? ee ie, Vole sijemhlk Sun. rele ave inahigie pOQyeatelehe = ery Gable, S04), 
if (r == 0) 
ynode->yrrules ++; 


Wie’ += YL; 


args.GetReturnValue().Set(Number::New(isolate, yrc)); 


void YaraNode::ScanFile(const FunctionCallbackInfo<Value>& args) { 
Isolate *isolate; 


Inia yrecan — 0; 


isolate = Isolate::GetCurrent (); 


HandleScope scope (isolate) ; 





NodeJS 


YaraNode *ynode = ObjectWrap: :Unwrap<YaraNode>(args.Holder()); 


Mieew(args. Lengun()w—-— Ie steargs | 0]-—-lseserimng())s eI 


AGt JUNE cpon aan wal cick a= 16) 


Cons Chart mle peti: 
Sinn YA 01010 (— ta ara an abel 1 ae OE 2 


Nae VMevouneni Sie Tejcre eileen (WaaelelS=e len (ie GNoe |) I HIIOIS (Sule 


SHEieal elope lUseumieWeeblible. watsyeuaty Weuae tse 0) are dkersneuaalvave pt) ) a: 
ieubd =e yoQelgie = Wimshnaa 
MMV OCNGE as Wie UL See poretshaip a LARS iene, ManMereuelel; ly 


NOHE ep, ANON. Le) 


} 
ergs. SeERSeurn Value ocr Numbers: New teoucbey soy nmicCal) s)\ry 
} 
void YaraNode::Init (Handle<Object> exports) { 
Local<FunctionTemplate> temp; 


Isolate *isolate; 


isolate = Isolate: :GetCurrent(); 
temp = FunctionTemplate::New(isolate, New); 


temp->SetClassName (String: :NewFromUtf8 (isolate, "YaraNode")); => 
From within a nodejs script, the class will have this name, we could have named it differently if 
necessary 





NodeJS 


NODE Zot PoP ROLOT Prev THOD (temp, — addnhule! ava raNnede = AVCdRulS)r, => As 
the single functions with NODE SET METHOD, we expose our methods via this macro 


INOIDIS NSN Mb Nes. OMOMRN aed Sicha slOLD eo “asters ialsIo A leche elelow wisiere) quel aldLoy) 


constructor.Reset (isolate, temp->GetFunction()); => Weclear the Per- 
sistent storage for each YaraNode instantation 


exports->Set (String: :NewFromUtf8 (isolate, "YaraNode"), 


temp->GetFunction()); 


void YaraInit (Handle<Object> exports) {f{ 


YaraNode::Init (exports) ; 


NODE MODULE (vara; 4 Yaralirte) 


temp->InstanceTemplate()->SetinternalFieldCount (2); 


We could test this module via this simple nodejs script. 


require('./build/Release/yaranode'); 


new sm.YaraNode(); 


eee a 


Ngchieuien Gi Aer cue lolkabWleay (Gia bAey Meas oy ey 





suey tee sa poiersng dials WG) s<amalili/=: Meysnmiales4 tier 


( erchercis, (pa, 4 


console.log (ex) ; 





This is a simple example and can of course be greatly improved but that might give you some 
ideas about the possibilities. On several known repositories, a significant amount of native nodejs 
projects already exist that use some popular components (like node geoip, for example). | hope 
this article is able to motivate you enough to start building your own nodejs modules. 
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Cloud service from a developer point of 
view 


In this article, we will have an overview of writing a cloud 

service. Various ways exist to achieve your goals but we will 

focus on one that is memory efficient, multiplatform (POSIX 

systems), multi language (from C++ to Erlang), and reasona- 
bly fast. It is Apache Thrift. | recently fully wrote a cloud serv- 
ice and it worked reliably. 


To illustrate this, we will make a basic remote file handler, 
the server is written in C++ and the client written in Python 
as an example. 


1. Describing the service 


Our server will be able to deliver three different services, listing files or directories, deleting or 
moving a file. Thrift is an IDL (Interface Definition Language) based framework, hence you de- 
scribe your service via an abstract generic language and the Thrift compiler will generate the nec- 
essary code per programming language. The basic Thrift types are all we find in common in all 
languages, byte, binary, integer (i16/32/64), double, boolean, string, some containers as hash- 
map, sets or lists. For those familiar with C and or C++ we can define an atomic file with a 
“struct”: 


The number means the index of the name's field. A file in a UNIX system can have several types, 
not necessarily a regular file but a device, a socket and so forth. 
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So, let's enumerate each type we might need to identify the files, again “a la” C/C++: 


What if we store some file attributes like the size, the permissions bits ... ? Thrift allows you to set 
a struct inside a struct without problems as you can see below: 
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Now, we can start to describe the three Thrift “services” as below; for the first, we would like to re- 
turn a map of files and for the sake of shortening, we “typedef” it as below: 


In addition, for our services we would like to throw an exception in case something went wrong. A 
Thrift exception is very similar to a struct: 


If we do not write the required keyword, a field is then optional. If you re not sure for future devel- 
opment that a field ought to be required, I'd suggest to leave it optional as the clients would stop 
working if the previously required field was suddenly optional in the server's side. 
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Above all of that we might need to customize the language namespace to organize and avoid con- 
flicts; for Java and C++ developers, for example, it is pretty well known. The namespace will be 
translated as well in the target language's logic: 


The first will produce the usual C++'s namespace as: 


Whereas the latter will make the eforensics/cloud Python module. 


In the end, the thrift file might look like this: 
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eee PA elie! 
LS aa 
i116 mask 
164 size 


SP ENAINONCE  SHeNeinitelsyl< 


SsErucE file { 
ith Ie IS ACkee INAS le 
neal Joe eeuey Scale ete ceu eit 14 


string name 


(EN Gla Gl 1e Bilele ceneyaMigle a aI ea inde ee iiol sae 


SPA SleNOne OIG) ie ies eo ele ous songs | 


ia i co meorore l= 


Zao acuemic o 


Sieve (ailetes serbia eee yallerey | 


alge) Jikol toys ieumengier ts abers) S siCl  asctoibaligiciel ts eicehole, \sicne lal) 
Bal Sy ep eli one mbiohah Rep.) 


throws 


(Gale 








2. Generating the code 


Once the service is defined, we can now use the thrift compiler like this: 


In the C++ version, we realize that a skeleton server was generated as well. We will use it to im- 
plement our service! 
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namespace :: ee ce aisle 
namespace :: Rar wighiaalernal ser ang oNaeneereler ie 
namespace :: Rie igliagilgi ls Ment GisroNabsNe lone aa 


Namespace |. apes cle aisles eee ec an) aman 


DOOs Eesha rceeuprG, 


namespace ::eforensics::cloud; 


Claes ard heres et varc Cancer ar ae bu clamp ie lic ce bee se iva corn we) 


eiule dives. 


eid S sis igaysal(ete pken@iehlona (7 


// Your initialization goes here 


sige) sieh ooriongl ia cukiers Clits (gino AksGekn (Ss | Maite bigieies MeLOig lene, wool A euena ale; sa eiclmlan | 


// Your implementation goes here 


Prinen (Merorcnsiess ls \ne. ); 
} 
Aeon Ee eUn nar Nelo ere tei Veo mise Keely Mislmigunialec: ere elm) aa) 
// Your implementation goes here 


Pigtie he kOtenc Lec mich il vay 





als GD ICE, aD aO ha in horibOrs) ail uiga COLON ANH ER etoD ots sl enemies gel scwG(G (exo) GiclEy tsiwiO ly rs) enanuiglon 
dst) 


// Your implementation goes here 


Praveen (ono rons le semvanyy) 


gh Cue (rembigu(IMeWe wenaelen aie gtekg — yxeliete hic) =| 
PE OOrLe =. 2090: 
slalehigiee Rene ig <imivley Merona malere) stenaielINoiae-s clgtenyvelldcuaa (ig vewin mally ex cue veer talengielouad().)l 


Shas WAt—Ve| MONG Al Sele lel loys (Onmy a 1818 Oe testes (aka esd eK ie i861) aia Oye! oro iO)16  (lahe Nel 
fod neuen nee 


Siiglehiai=le Pema ibs! ney <= igdiiarcighsjeleaier a as(oey/doniailnigtoielsjele nals deteu) wily) <a (eiers @1G ce 
SiS crena che 


shared purl Transport Pacuory transporter aceory (new TBurreredirans— 
DOrEPactory ())¢ 


eilaedypVel tr rOrcOce EAacrOLy (UreroceMAceOuy New elias yr roroce lac — 
ore One: 


TSimpleServer server (processor, serverTransport, transportFactory, 
(Sigs elec oul Rereiwieng 2) e 


server.serve(); 


return OQ; 








Now it is up to us to implement the three services. Let's start with the simplest, removing a file 
with the famous C function unlink. 


To improve it, we could make sure the file is a regular file, otherwise return the exception we set 
earlier in the thrift IDL file. 
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meso t="pacth; 
1 ee Se ele Olen Gaal) 
ae  SVels sien Gilsien)s: 


jah a) tee 


SEMUCEr Sears, 
Creare Stal (s,) peach), 


MN OKS LEV = for Se) iieeleys 


if ((m & S IFMT) != S IFREG) { 


string msg = "Only files can be removed"; 
igen SN Siets eloleisdFauls jy: 
1 Bae ele jiliste (Gqiislen) 1 


BISA hale me 


dese bh GelaUeabicl <i jerenelenen gs ones kyl Realy ly a 
string msg = "Could not remove "; 
ks(6 (iar yekenel ai 
msg += 


msg += strerror(errno) ; 





peer Gn elorelon Gadly i 
ibe p Siele. ilstes Galisten) 


inlanaelin ae 


abi ge IO. 15) ereonals)qsiWerei iia (efsiqusne« Hsochtt swencaligleks woncle = Wel sig sis) IsnelCl ue Suenenli@lensa selene) 


{ 


SUSMO CRS) coer uemusy: 
(obal <Yol i ion e Wea (ou mecca eu) 


Olea Sai syst eey Ocha, 


sea AGI SS) CUBR) eee ea 
string msg = "Only files can be moved"; 
nF Me cial me eeiele (anh) 
Heres iaa ees sto Guise) ir 
es gia @) any 


} 


aan, Mig v aaNet Gsnaienyie ts) Chal) avteksn ae, Mosera (Ve aby 


String meql—) Couldenou move, 
msg += src; 


(NSN fe cyte ge ON ale 








Then the last service, listing files or directories. Previously, we defined several types of files and 
their attributes, hence we'll once again rely on the stat function: 
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fl.name = path; 


if ((m & S IFMT) == § IFBLK || (m & S ITFMT) —— § IFCHR || 


& S IFMT) == S IFIFO) { 
16 th IES Tesla ee Aston DIN Glee 
& S IFMT) == § IFSOCK) { 
Holds ISK ASlo SEIS yee sO ImMae 
& S IFMT) == § IFLNK) { 
Bile eee ae yoe oni 
& S IFMT) == S_IFREG) { 

cade, aA Se IG ACle Bes Nels rare RII) hy: 


eet & S IFMT) == S IFDIR && (((m & S IFMT) & S_IFLNK) 
!= § IFLNK)) 


jee elon) igi R Cee icici en aeic ss DIM ee NUOls ce 


iS eee elie 


pal rel Bamiers ered Fe (0)c 
if (m & S_TWUSR) 
fl.attr.mask |= 0x400; 
if (m & S _TRUSR) 
EAL Peseisia ese <j (p.c2 Ol0)e 
sm (Ot Se Se AOrSIRS) 


flvattr mask |— Uxl00; 





ala 


(m & S IWGRP) 


fall wens Se erlensle Ox040; 
Guana INS C@350=) 
pH IL Re NE Gla A litolod.< 
(cana RNID. CO)=0-D) 
iB llgageh we isnaqaiiirecut Opener 
(m & S IWOTH) 
fie eieiiack 0x004; 
(m & S TIROTH) 
Ps ape mials kK 
Greer aes Oui) 2b. 


Eater .mask Ose O Oe: 


ol gisngws i aiitises) eer ||p 
SHON MoMErn @Shena iksy eae toll Gak Man ll wolsaseeegtitolsn 2) ie 


PIR Aker see rias Ke noe ec e elas ky). 


igeisionaial|Meovsietey|) eon ley: 

obo ai Gal cuDebwas yee 
BARS relia 1s) cioolaiae ierelelona ey se ea lL) 
if (dir == NULL) { 


reburn —— 1s 





SLrUuct dirent entry, *resnul. = NULL; 


// Ne could have just used readdir but we might need to run it 


// in multi thread context .. 
(gt Sal ial teh o lo Ree ion QolRiay ye Signs ianya, a Ssia fs sbile) ys = 100 he 
if (result == NULL) 
break; 

Ie, aps eanerien hae Yo aecrsibel ey eh Valente) as 20) 8 
SHENSON BAe Mister Ole eel Saveniite) 1) 
continue; 

SHEIaahighe@ majel-usial ay jer-melaie 

Gish alelsuulak berselatencniva 0G) aciip. ae 2) 
Poeathet— y's 

ia) ele OIG Wea Na rw eee cel alelittor 


ISeacdrenery (Weel mn, mCP achy) 7 


elerse(enbian Genie es 


return O; 








We are nearly done, let's compile the code. 


If you execute the final executable, it will listen via the 9090 port and if you generated Python's 
version, for example, it should have generated a sample client: 
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{ '/tmp': file(type=4, attr=file attribute(gid=0, mask=None, uid=0, 
strmask='0777', size=None), name='/tmp'), 


'/tmp/.ICE-unix': file(type=4, attr=file attribute (gid=0, 
mask=None, uid=0, strmask='0777', size=None), name='/tmp/.ICE-unix'), 


‘/emp/.ICE-unix/1997": file(type=2, attr=file attribute (gid=1000, 
mask=None, uid=1000, strmask='0777"', size=None), 
igyctitlom = eloy ey INOld olga aI Soy Lae 


"/tmp/.XO-lock': file(type=0, attr=file attribute (gid=0, mask=None, 
uid=0, strmask='0222', size=None), name='/tmp/.X0-lock'), 


"/tmp/.X11-unix': file(type=4, attr=file attribute (gid-=0, 
mask=None, uid=0, strmask='0777', size=None), name='/tmp/.X1ll-unix'), 


'/tmp/.X11l-unix/X0': file(type=2, attr=file attribute (gid=0, 
mask=None, uid=0, strmask='0777"', size=None), 
name='/tmp/.X11l-unix/X0"'), 


Wy SMG <5 Vdelon.<teleiclialligetemaljeve We ar mikey (sere 7A wecns sia —oe INS el sisiadli= 
bute (gid=1000, mask=None, uid=1000, strmask='0700', size=None), 
name='/tmp/.vbox-dcarlier-ipc'), 


“7 emp/ .vbox-dearlier—-ip]e/iped" <  file(type—2, ater—Tile atcri— 
bute (gid=1000, mask=None, uid=1000, strmask='0700', size=None), 
hame="/tEmpy -voex-dearlier—-ipc/iped'), 


'/tmp/.vbox-dcarlier-ipc/lock': file(type=0, attr=file attri- 
bute (gid=1000, mask=None, uid=1000, strmask='0600', size=None), 
hamne—' jmp, ,7oOx-dcarliet—ipe/ lock); 


‘/tmp/config-err-tu3hNl': file(type=0, attr=file attri- 
bute (gid=1000, mask=None, uid=1000, strmask='0600', size=None), 
ighs h(a ey By Mele) glibale fetta aoetelbls al Nua) as 


"/tmp/unity support test.0': file(type=0, attr=file attri- 
bute (gid=1000, mask=None, uid=1000, strmask='0662', size=None), 
name='/tmp/unity support test.0') } 





ss eal ge 0 oh On 
brane pOorre— THErOC Lich e THE OCiIenn (host, oor Ee, aura.) 


Sesto 


socket = TSSLSocket.TSSLSocket (host, port, validate=False) if ssl 
else TSocket.TSocket (host, port) 


if framed: 
# In this mode, the message is fully read no flush is required 
Sa SIOKS See |e TTransport.TFramedTransport (socket) 
else: 
ipiccinie OC ae TTransport.TBufferedTransport (socket) 
[ciao se cole IisaRelsnandennense colby iWin ak glioma deiaenme) eo iL Gena cngls\ closets) 
Mie iaiz y= nouivlitc isi ones elon © Ihnen (shaoe/oreroul,) 


(Euashevs Cloner yy Ol eroielt |) 


# Pretty straightforward to call each server method as you can see 


pe (OU Ch ert une ie cighs aliens ellysks & 
if len(args) != 1: 
jQuenligie (( Wounonasiavculictse Wlis's ameeiuliiesics Jl keligers.}) 
Syvs-exit (1) 


[S/O S) Seana en Vodka igi ste cue ea cialculence lac veuceice Oh y7i))) 








If we come back to the C++ server's code, the skeleton's generated code uses a TsimpleServer, 
which is perfect to start with but is monothread. I'd suggest the TThreadPoolServer (more effi- 
cient than the TThreadedServer) or the TNonBlockingServer instead and to at least add a signal 
handler to terminate the server properly. The TthreadPoolServer's version might look like this: 
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sie pele ECS IMEC IEE si Sigsiesiake lavenelellcng)) & 


Signal (SIGPIPE, servsighandler) ; 


ae aa 


SU GelidiarOl FOS ie <MWiiaelersieyelonar, Jehcelclacieiona (igo (6 lowiel Ta araiers sevol tah q encore 
srewsroug a latetselliicia)) ir 


Saree eDeE ect Cerra ispOrt sch vCh ME anopOLe (ewe loch VetoOCh = 
ed Gelenasen py) 


sMevelial Vel] gon sian sie chalsy el enaie iene wonane. Fe lec asio@ ia .e iene won elmo; Mnotbioue rales 
Mean SOOrE meu tr 7) ts 


Sialclie 16 /eoneie, GIhligensiel@lorlisccleisene ee Nels eheeloed! Melenteus va igioi (ple eli gtalie V/s O1s(Oles 
colFactory()); 


threadManager = 
ThreadManager: :newSimpleThreadManager (workers) ; 


Sleicume Cl) jeisiacieiorsuin a Mgmecvcrc lM cveretenaysc. clobiaiercic INKS elonayel sles Maersily.<ciMawecrclon 


ere monarchy) es 
threadManager->threadFactory (threadFactory) ; 


threadManager->start (); 


Sumio le @sllolo( <0 Uisloua sou als) st mca leah gleni™ «Co ieee crave len 


HSserver — shared per<loerver- (new TihreadPooblerver (proces— 
SOL, Sserverlranspore, EeranspOorEerFaAcEory, DrOnOCcOolrPactonuy, elireadMan— 
elefeie)) 1) e 


nserver->serve (); 


iImCak Ch WISE dt ex Cepedlone ce) rn 





SE Olea elef e UW Nghe casi goucsas Cena chee ee “oun eleua) eG. oi me lees cu ell ly: 





3. Conclusion 


Apache Thrift works well indeed in most POSIX systems, I've made the full example server part 
of a Linux machine and tested with FreeBSD and Linux. The client was called on a remote 
FreeBSD machine. 


An alternative version exists remade by Facebook called fbthrift which works fully only on Linux 
but the code generated is superior and this version in general has proved to be more efficient in 
term of memory usage at least. There is also Google Protocol Buffer that performs better than the 
two above but has fewer languages supported (officially). You have to write the client / server 
code on your own, though. So based on your own criteria and restrictions, one of these might fit 
better for your own case. 
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